TITLE OF THE INVENTION 
Cooperative Application System, Cooperative Application Method and 
Network Terminal 

BACKGROUND OF THE INVENTION 
Field of the Invention: 

This invention provides a cooperative application system and network 
terminal that links the operation of applications used by presenters at 
a multimedia conference with the applications of the participants at the 
conference. 

Description of the Related Art: 

Currently, various kinds of communication services are provided by 
different communications businesses to general consumers and business 
users. One of these services is a audio conference server that connects 
normal telephones, portable telephones, PHS phones and the like that are 
located at different locations and allows simultaneous conversation (for 
example a chorus line service). 

On the other hand, with the increasing size of the Internet market, 
text -conference services and text-chat services are also provided that 
allow users to share data- type information such as text data, image data 
and music data over the Web with many users (for example, a Web conference 
room) . 

In network -provided conference services such as the audio -conference 
services or text-conference services described above, it is desired that 
real-time audio -conference services and data-type services be mutually 
linked together based on user needs and diversification of the type of 
business . 

Moreover, in recent years, due to the advancement of VoIP (Voice over 
IP), Internet telephone applications that make two-way, real-time audio 



communication possible over an IP (Internet protocol) packet network such 
as the Internet are beginning to become popular. 

A prior system that provides a multimedia conference service that 
links the operation of data-type conferencing and real-time audio 
5 conferencing comprising at least an image-information-sharing unit, 
white-board- sharing unit, speaker-screen-display unit , 

arbitrary-information-search unit, arbitrary- information- notice unit, or 
conference-minutes-creation unit has been made possible. Also, for audio 
conferencing, multimedia-conferencing services are provided that combine 

10 a audio-communication terminal for circuit switching, such as a telephone 
terminal, and a VoIP communication terminal, such as an Internet -telephone 
terminal (for example, refer to Japanese unexamined patent publication 
No. 2001-292138 (paragraphs 1 to 7 , and Fig. 2)). 

Typically, in the case of performing communication using a shared 

15 transmission media type of network, the amount of information that can 
be transmitted by a terminal at a certain time varies according to the 
amount of information that is transmitted by another terminal during that 
time. Therefore, when a terminal is attempting to transmit data, it is 
not possible to guarantee that all will be transmitted without delay. 

20 Actually, in the case in which the output video of the application that 
is displayed on the terminal on the presenter 1 s side is transmitted together 
with the video and audio of the presenter, the amount of data to be 
transmitted becomes very large so problems may occur such as audio data 
not matching the video data, or delays in transmitting data due to an 

25 increase in the amount of traffic. In other words, on the receiving side, 
when there is a delay in the display of the image with respect to the audio, 
or when the image is broken up, or when it is clearly evident that the 
audio (conversation) of the presenter is delayed, it becomes difficult 
to maintain a conversation. 

30 Particularly, in the prior technology, video of the documents 
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presented by a plurality of users located at different locations was 
provided by transmitting the video presented by the user of the terminal 
at the transmission source to a plurality of terminals on the receiving 
sides that were connected over the network. Also, at the same time as this, 
5 the video of the presenter, for example, was transmitted as a moving image. 
Therefore, there was a problem in that each time the video of the documents 
being presented changed, the video of the documents being presented had 
to be sent to the receiving side, and whenever the video of the documents 
being presented changed, the video of the presenter and documents was broken 

10 up due to the increase in traffic. In order to deal with this problem, 
the next generation Internet protocol IPv6 (Internet Protocol Version 6) 
is being installed. This new version IPv6 is based on the current IPv4 
( Internet Protocol Version 4 ) , and is improved by having an increased amount 
of address space, added security features, and data transmission according 

15 to priority, and it makes it possible to perform control such that AV packet 
information from network cameras (or digital cameras) is sent or received 
having priority. 

However, particularly in the case of providing the 
multimedia-conference service described above, it is necessary that both 

20 the computer, which is used near the screen that uses text, and the home 
television, on which moving images are watched away from the screen, have 
adequate resolution, and a resolution of 2 M-pixels/f rame is necessary. 
This corresponds to the HDTV 1100 that is shown in Fig. -11, or in other 
words, corresponds to high-definition (HD) quality. From now on, this kind 

25 of high-definition quality will become the norm, and when it further becomes 
possible to frequently send and receive a plurality of AV information, 
naturally the amount of traffic on the network will increase. Therefore, 
it may not be possible to completely solve the problems related to the 
amount of network traffic even when using the IPv6 that gives priority 

30 to sending AV information. 
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Furthermore, there are also problems in processing the data on the 
receiving side. That is, when switching videos at the terminal on the 
sending side, processing is performed by the terminal on the receiving 
side to switch the video from the video before the change to the video 
after the change, so the amount of internal processing performed by the 
terminal on the receiving side increases, and a problem exists in that 
the sound ceases because sequential processing of the sound by the VoIP 
becomes impossible. From this aspect as well, it is predicted that this 
will become an even larger problem due to the increase in the amount data 
for changing to high definition quality. 

It is the object of this invention to provide a cooperative application 
system and network terminal that is capable of displaying and reproducing 
the video and audio of a presentation at a location that is separated from 
the location of the presentation without a break up in the video or sound, 
and further is capable of reducing the load of internal operation processing 
of the terminal. 

SUMMARY OF THE INVENTION 

In order to accomplish the aforementioned object, this invention 
20 adopts the following means. That is, it is presumed in this invention that 
the cooperative application system links the operation between a terminal 
on the sending side with a terminal on the receiving side that is connected 
by way of a network. Here, the terminal on the sending side comprises: 
a first application-control unit that outputs instructions to the 
25 application operating at the terminal on the sending side, and a sending 
unit that sends the instructions that were output from the first 
application-control unit to the terminal on the receiving side. In other 
words, the first application-control unit sends instructions to the 
application that is operating at the terminal in which it itself is mounted, 
30 and the sending unit also sends that instruction to the terminal on the 
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receiving side. Moreover, the terminal on the receiving side comprises: 
a receiving unit that receives the instructions from the terminal on the 
sending side; and a second application-control unit that outputs the 
received instructions to the application that is operating at the terminal 
5 on the receiving side. 

By doing this, it is possible to link to and display the presented 
documents at the terminal on the receiving side without sending the video 
data of the presented documents that are output by the application of the 
terminal on the sending side, so it is possible to reduce the amount of 

10 traffic on the network and to lighten the burden of the operation processing 
by the terminal on the receiving side, and further it is possible to output 
the video and audio of the presenter together with the video of the presented 
documents that are output by the application to a multiple of locations 
that are separated from the location of the presentation. 

15 Also, at least either the terminal on the sending side or the terminal 

on the receiving side comprises an application-data-management unit that 
checks the compatibility between itself and at least the type of application 
that is operating at another terminal, the application status at another 
terminal, or the application data that is used by the application at another 

20 terminal. 

With this construction, it is possible to avoid trouble such as when 
the linked operation of applications become strange when the application 
data used by the application on the side of the presenter differs from 
the application data used by the application of another participant of 

25 the conference. 

Also, the sending unit sends address information for the terminal on 
the receiving side, contents to be used by the application that is operating 
at the terminal on the receiving side, and sending instructions to a 
specified server for sending the contents to the terminal on the receiving 

30 side; and the receiving unit receives those contents from the server and 
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gives them to the application that is operating at the terminal on the 
receiving side. 

Furthermore, the sending unit sends the contents that are to be used 
by the application operating at the terminal on the receiving side to a 
5 specified server, and sends the address information of that server to the 
receiving unit of the terminal on the receiving side; and the receiving 
unit receives those contents from the server according to the received 
address information for the server, and gives the contents to the 
application that is operating in the terminal on the receiving side. 

10 By doing this, the terminal on the sending side is able to send the 

contents for a conference or the like to a server at an appropriate time 
before the contents become necessary, or in the case in which the contents 
to be used are already stored in the server, it is able to obtain those 
contents at an appropriate time according to conditions at each location. 

15 Also, the terminal on the receiving side comprises a first time-control 

unit that synchronizes the video signal that was input to the video-input 
unit, the audio signal that was input to the audio-input unit, and the 
instructions that were output from the application-control unit and 
outputs them to the sending unit; and the terminal on the receiving side 

20 comprises a second time-control unit that receives the synchronized video 
signal, audio signal and instructions, and synchronizes and output the 
video, audio and instructions. 

With this construction, by performing synchronization, it is possible 
to lessen the problem of delays in transmitting data due to shifts in audio 

25 data and video data or increases in the amount of traffic. 

When the video signal that is input from the video -input unit is a 
high-definition video signal, the amount of data that the video occupies 
becomes large, and further, since a large amount of processing by the 
terminal on the receiving side becomes necessary, the effect of this 

30 invention is very evident. 
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Here, the cooperative application system and network terminal can be 
embodied using a computer. In that case, each of the units described above 
are embodied by operating a program on a computer. 

With the cooperative application system, cooperative application 
5 method and network terminal of this invention, copy data of the documents 
presented by the user on the sending- terminal side is also set at the 
receiving- terminal side, and by operating the application, the video 
signal that is output from the application at each of the locations is 
switched according to an application- output -switch instruction from the 
10 user of the sending terminal or preset conditions. As a result, it is 
effective in reducing the amount of network traffic, lightening the load 
due to operation processing by the terminal on the receiving side, and 
outputting video and audio of the presenter together with the 
application-output video of the presented documents without breaking up. 

15 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic diagram for explaining the cooperative 
application system of a first embodiment of the invention. 

Fig. 2 is a block diagram showing the construction of the cooperative 
20 application system of the first embodiment of the invention. 

Fig. 3 is a drawing showing the flow of the processing by the cooperative 
application system of the first embodiment of the invention. 

Fig. 4 is a block diagram showing the internal construction of the 
application-control unit of the first embodiment of the invention. 
25 Fig. 5 is a schematic drawing for explaining the cooperative 

application system of a second embodiment of the invention. 

Fig. 6 is a block diagram showing the construction of the cooperative 
application system of the second embodiment of the invention. 

Fig. 7 is a drawing showing the flow of the processing by the cooperative 
30 application system of the second embodiment of the invention. 
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Fig. 8 is a block diagram showing the construction of the cooperative 
application system of a third embodiment of the invention. 

Fig. 9 is a block diagram of the time-control unit of the third 
embodiment of the invention. 
5 Fig. 10 is a timing chart for the video signal, audio signal, 

application-control signal and synchronization- signal of the cooperative 
application system. 

Fig. 11 is a drawing showing the amount of operations and the 
transmission speed required for encoding and decoding digital video. 

10 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The preferred embodiments of the invention will be explained in detail 
below using the drawings. 
(Embodiment 1) 

15 Fig. 1 is a schematic drawing for explaining the cooperative 

application system in a multimedia-conferencing system that uses the 
network terminals of this invention. 

The network terminals 100, 200, 300 are used as user terminals in the 
multimedia-conference system. Also, network terminal 100 is the terminal 

20 used by the presenter and is located on the side of the presenter, and 
network terminals 200, 300 are located at the locations of the participants 
who participate in the conference. Hereafter, when the terminals are 
simply referred to as the participants, it will be assumed that the 
presenter is not included. 

25 The network terminals 100, 200, 300 at each location are respectively 

connected to projectors 400, 500, 600 that receive video signals that are 
output from the network terminal, and project the received video on 
respective screens 800, 900, 1000. 

In this invention, the network terminal 100 on the presenter's side 

30 outputs the application data of the presentation documents as a video signal 
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to the projector 400, and switches between video signals to be output to 
the projector according to instructions from the presenter or according 
to a preset condition. The presenter instructions referred to here are 
instructions from the user that is actually using the video projected on 
5 the projector to the application that is outputting the presentation 
documents to switch screens. Also, the preset condition referred to here 
is switching timing that is set for the application that outputs the 
presentation documents, and it is preset for the application by the user 
to automatically switch the output video. 
10 The network terminals will be explained here using Fig. 2. Fig. 2 

is a block diagram showing the construction of the cooperative application 
system. 

In Fig. 2, the network terminal 100 can be embodied by connecting it 
to the units described below by way of a bus 112. 
15 The application-operation unit 101 reads the presentation documents 

used in the conference and converts them to video data, and operates the 
application. 

The instruction- input unit 110 is a unit that allows the user to input 
instructions to the terminal such as an instruction to switch the video 
20 that the application outputs, and it corresponds to the mouse or keyboard. 

The application-control unit 102 outputs instructions to the 
application to perform specified operations according to instructions 
input by the user from the instruction-input unit 110 or according to a 
preset condition. 

25 When the instruction that is output from the application-control unit 

102 is sent from the network terminal, the sending unit 103 sends it to 
a network terminal that is used by another user (for example, network 
terminals 200 , 300 ) . 

When the user of another network terminal (for example, network 

30 terminal 200, 300) sends an instruction to an application that is operated 
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by another network terminal, the receiving unit 104 receives that 
instruction. 

The I/F processing unit 105 connects to the network in order to perform 
communication with the peripheral devices of the network terminal 100. 
5 The IP-audio-conversion unit 106 performs conversion of IP packet 

signals and audio signals for sending audio in both directions in real-time 
over an IP packet network such as the Internet. 

The speakers 107 output the audio signal that was converted by the 
IP-audio-conversion unit 106. Also, the microphone 108 inputs audio to 
10 the IP-audio-conversion unit 106 as an audio signal. 

The video-output unit 109 outputs the video signal that is output from 
the application that is being operated by the application-operation unit 
101 to the display unit that is connected to the network terminal 100. 
The address-setting unit 111 receives input for the address of the 
15 terminal and the address of the destination terminals, and stores the input 
addresses . 

The application-data-management unit 115 checks the version and status 
of the applications operating at each terminal, the compatibility of the 
read application data, and like. 

20 The network terminals 200, 300 are network terminals having the same 

functions as the network terminal 100. Reference numbers 201 to 211 of 
network terminal 200, or numbers 301 to 311 of network terminal 300 
correspond to the numbers 101 to 111 of network terminal 100, and since 
the operation is the same, an explanation will be omitted here. 

25 The projectors 400, 500, 600 are connected to the network terminals 

100, 200, 300, respectively, and they receive the video signal from the 
video-signal-output units 109 , 209 , 309 and project the video on the screens 
(not shown in Fig. 2) . In other words, they function as the display units. 
Next, the operation of the cooperative application system will be 

30 explained using Fig. 1, Fig. 2 and Fig. 3. Fig. 3 is a flowchart showing 
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the flow of processing by the cooperative application system. 

Here, as shown in Fig. 1 and Fig. 2, network terminal 100 is set up 
at location A, network terminal 200 is set up at location B and network 
terminal 300 is set up at location C. Network terminal 100 at location 
5 A, network terminal 200 at location B and network terminal 300 at location 
C are connected such that they can communicate with each other over network 
700. Also, when one of the network terminals calls another, and the user 
of the network terminal that receives the call responds to the call, two-way 
audio communication becomes possible. In order to simplify the 

10 explanation below, a conference between two location, location A and 
location B, will be explained. 

First, the application operated by the application-operation unit 101 
of network terminal 100 reads application data as the presentation material 
used in the presentation. Also, the application-control unit 102 sends 

15 an instruction to the application based on an instruction from the user 
or based on a preset condition, and as a result the application switches 
the video signal to be output to the projector 400. 

Also, electronic files, which are presentation documents, are sent 
from the user to the participants in the conference in advance, and the 

20 participant at each of the locations (for example, location B) reads the 
electronic files that were received in advance into the application of 
the network terminal 200, and the same condition as the presenter, or in 
other words, the standby state is set (Fig. 3: step S101) . The applications 
used at location A and location B are the same. 

25 At this time, the application-data-management units 115, 215, 315 of 

each of the network terminals can be such that they check the communications 
as to whether or not there are any differences in the electronic files 
that set the status to standby and that are used in the conference. For 
example, the application-data-management unit 213 makes an inquiry of the 

30 application-data-management unit 113 to check whether or not information 
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related to the presentation documents, such as the file names and dates 
of creation of the files used in the conference, is the same. By doing 
this, it is possible to avoid problems such as differences in the 
presentation documents activated by the application on the side of the 
presenter and the presentation documents activated by the application on 
the side of the participant, and abnormal operation of the cooperative 
application system. Moreover, at the same time, the 

application-data-management unit can check whether or not the applications 
used and versions of those applications are the same, or whether or not 
the status of the applications is the same. The status referred to here 
is whether the application is in the input-ready state (whether there is 
focus), and whether the display size of the application is within a 
specified range. By performing this check, it is possible to assure proper 
operation of the cooperative application system. 

Next, the control modes of the network terminals used at each location 
are set. For example, the network terminal 100 that is used at location 
A on the side of the presenter is set to the mode to receive instructions 
(application-control signals) from the instruction-input unit (normal 
control mode). The network terminal 200 that is used by another 
participant in the conference (for example, at location B) is set to the 
mode to receive instructions from the outside by way of the receiving unit 
(remote-control mode) (Fig. 3: step S102). These settings are performed 
using a control-mode signal from the instruction-input unit. 

Here, Fig. 2 and Fig. 4 will be used to explain the differences in 
the operations of the application-control units 102, 202, 302 according 
to the control mode. Fig. 4 is a block diagram showing the internal 
construction of the application-control unit. 

The first cooperative application unit 102A receives an instruction 
from the instruction-input unit 110 in order to link operation with the 
external application of network terminal 100, and sends it to the sending 
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unit 103. 

Also, the control-instruction-selection unit 102C selectively outputs 
the instruction from the instruction-input unit 110 or instruction 
received by the receiving unit 104 based on the preset control mode. In 
5 other words, when the control mode is set to the normal control mode, the 
instruction input from the instruction-input unit is output to the 
application-operation unit, on in other words the application, and when 
the control mode is set to the remote -control mode , the instruction received 
by the receiving unit is' output to the application. 
10 The second cooperative application unit 102B sends the instruction 

received from the control-instruction-selection unit 102C to the 
application operated by the application-operation unit. 

Units 202A, 202B, 202C of network terminal 200, or units 302A, 302B, 
302C of network terminal 300 correspond to units 102A, 102B, 102C of network 
15 terminal 100 and the operation is the same. Therefore, an explanation of 
units 202A, 202B, 202C, and units 302A, 302B, 302C will be omitted here. 

The flow of the processing by each unit is described below. 

The network terminals 100, 200 comprise address- setting units 111, 
211, and by setting the address of the main unit, it becomes possible to 
20 perform two-way communication by simply connecting to the network. In 
other words, the user of network terminal 100 at location A specifies the 
communication destination (here this is the network terminal 200 at 
location B), and connects to that terminal by way of the network. On the 
other hand, when the connection destination receives that connection 
25 request, two-way communication becomes possible (Fig. 3: step S103 to step 
S104) . 

In the state where two-way communication becomes possible, it is 
possible for the user at location A to perform audio communication with 
the user at the other location (for example, location B). Also, in this 
30 state, the network terminal is in the standby state waiting for a control 
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instruction for the application (Fig. 3: step S105). 

The presenter sends an instruction to the network terminal 100 from 
the instruction-input unit 110 to switch the application output video (or, 
sets in advance the switching timing for the application, and sends an 
5 instruction to switch the application output video based on that switch 
-timing setting) (Fig. 3: step S106) . The main instruction needed for the 
application at the conference locations is for changing the video that 
the application outputs, however, switching the application output video 
referred to here includes all of the instructions for the application. 
10 In other words, the same occurs even when instructions other than the 
instruction to switch the output video are sent to the application. In 
other words , the output video from the application is not included in the 
instruction. 

When the application-control unit 102 of the network terminal 100 

15 receives an instruction from the presenter, it sends that instruction to 
the application-operation unit 101. Also, the application-control unit 
102 sends the instruction by way of the sending unit 103 to the network 
terminal 200 used by another participant in the conference (for example, 
at location B) (Fig. 3: step S107). 

20 The network terminal 200 receives the instruction that was sent from 

the network terminal 100 on the side of the presenter by way of the receiving 
unit 204, and then sends it to the application-control unit 202 of the 
network terminal 200 (Fig. 3: step S108). 

The application-control unit 202 sends a control instruction to the 

25 application to switch the output video received from the other application 
(Fig. 3: S109). In this way, the video output from the application on the 
side of the network terminal 200 is switched according to the instruction 
from the network terminal 100 at location A, and as a result, it is possible 
to switch the video output from the application at location B together 

30 with switching of the video output by the application at location A. 
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As explained above, with this invention, a copy of the data of the 
presentation documents presented by the user of the network terminal on 
the side of the presenter is also set in the network terminal on the 
receiving side and the application is operated, and the video signal that 
5 is output from the application at other locations is switched according 
to an instruction from the user of the sending network terminal to switch 
the output from the application, so it is not necessary to send a large 
quantity of data to the receiving side. As a result, it is possible to 
reproduce the same video as the presented video at a location separated 

10 from the location of the presentation without the video breaking up and 
without delays. Also, the data received for the video is just an 
instruction to the application, so it is possible to lighten the burden 
of operation processing inside the terminal. Naturally, the applications 
at each location, the status of the application and compatibility of the 

15 application data are ensured by communication between 
application-data-management units . 

The audio signal during the conference is obtained by the network 
terminal from the microphone 108 (208, 308) of the network terminal at 
each location, and the IP audio -conversion unit 106 (206, 306) converts 

20 it from an audio signal to audio IP packet data. Here, sending audio will 
be explained using the audio signal obtained by the microphone 108 of the 
network terminal 100 that is sent to the network terminals 200, 300 at 
other locations as an example . The audio IP packet data that was converted 
by the IP audio-conversion unit 106 of the network terminal 100 is sent 

25 to the other network terminals (for example, network terminal 200, 300) 
participating in the conference other than the network terminal (for 
example, network terminal 100) that sends the audio IP packet data from 
the sending unit 103 via the network 700. The network terminals (for 
example, network terminals 200, 300) that receive audio packet data, 

30 receive the audio packet data by way of the receiving unit (for example. 
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204, 304), The audio packet data that is received by the receiving unit 
is converted to an audio signal by the IP audio conversion unit (for example, 
206, 306), and reproduced by the speakers (for example, 207, 307). Also, 
in the case where the receiving unit of one network terminal 200 received 
5 a plurality of audio packet data, the audio packet data is converted by 
the IP audio-conversion unit 206 into an audio signal, and then a combining 
process can be performed. By doing this, it is possible to reproduce in 
real-time the audios of a plurality of conference participants. In 
addition, since switching the application data is possible by simply 

10 sending and receiving display switching instructions as described above 
instead of sending display data as done conventionally, multimedia 
conferencing that is more efficient and economical than conventional 
real-time telephone conferencing is possible. 

Moreover, in the explanation above, an example of switching the 

15 application output video was used for explaining the operation of linking 
the application on the side of network terminal 100 with the application 
on the side of the network terminal 200, and switching the output video, 
however, this invention is not limited to this, and it is also possible 
to link other operations by sending and receiving instructions for 

20 similarly controlling the operations of network terminals at different 
locations. 
( Embodiment 2 ) 

In a second embodiment of the invention, a cooperative application 
system is explained in which contents are stored on an Internet server, 
25 and when the contents are necessary, a request is sent from a terminal 
to the server to send the contents. 

Fig. 5 is a drawing for explaining the cooperative application system 
of this second embodiment of the invention. 

Also, Fig. 6 is a block diagram shown the construction of the 
30 cooperative application system of this second embodiment of the invention. 



16 



Network terminals A100, A200, A300 are network terminals on the side 
of the users of a multimedia conference system. Also, network terminal 
A100 is located on the side of the presenter, and network terminals A200, 
A300 are located at each conference location. Moreover, each of the 
5 network terminals A100, A200, A300 have the same construction and same 
functions as the network terminals 100, 200, 300 in the first embodiment, 
and so any redundant explanation of them will be omitted here. 

The server A800 receives a send-contents request from a network 
terminal and sends the contents to each network terminal, and this is made 
10 possible by connecting each of the units described below via a bus 806. 
The server A800 as shown in Fig. 6 has only the elements necessary for 
this second embodiment . 

The contents-storage unit 801 stores contents that were received from 
the outside via the network 700 or a portable recording medium. 
15 The user-information-management unit 802 manages the users that send 

contents such as conference documents to the server, and users that receive 
contents from the server. 

The receiving unit 804 receives contents from an external network 
terminal, or receives instructions to distribute received contents to a 
20 specified external network terminal. 

The sending unit 803 sends contents to a specified external network 
terminal based on a contents -distribution instruction received by the 
receiving unit 804. 

The I/F processing unit 805 performs the connection process for 
25 connecting to the network in order to communicate with devices outside 
of the server A800. 

Next, Fig. 5, Fig. 6 and Fig. 7 will be used to explain the operation 
of the cooperative application system of this second embodiment. Fig. 7 
is a flowchart showing the flow of processing performed by the cooperative 
30 application system of this second embodiment. The processes that differ 
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from those of the first embodiment will be explained here. As in the first 
embodiment and as shown in Fig. 5 and Fig. 6, the network terminal A100 
is located at location A, network terminal A200 is located at location 
B, and network terminal A300 is located at location C. The network terminal 
5 A100 at location A, the network terminal A200 at location B, and the network 
terminal A300 at location C are connected over a network 700 such that 
they can communicate with each other. In order to simplify the explanation 
below, an example of a conference between the two locations, location A 
and location B, will be explained. 

10 The application being operated by the application-operation unit 101 

of the network terminal A100 reads the documents used in the presentation 
beforehand. Also, the application-control unit 102 sends instructions 
that are output to the application based on an instruction from the 
presenter, or based on a preset condition. As a result, the application 

15 switches the video that is output to the projector (not shown in the figure) . 
Also, the sending unit 103 of the network terminal A100 sends the contents 
to be used in the presentation to the server A800 (Fig. 7: step S201). 

Next, the server A800 receives the contents from the network terminal 
A100 via the network 700, I/F processing unit 805 and receiving unit 804, 

20 and stores the contents in the contents -storage unit 801 (Fig. 7: step 
S202) . 

The method of receiving the contents is not limited to receiving 
contents from the network terminal via the network, for example, it is 
also possible to receive the contents via a portable medium. Also, the 

25 contents are not limited to being the presenter's documents on the side 
of network terminal A100, for example, the contents could also be documents 
that were created by some means other than by the network terminals A100, 
A200, A300. Needless to say, in this case, it is necessary for the rights 
to use the documents by the network terminals be given for contents that 

30 were created by other than the network terminals. 
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Also, after the network terminal A100 calls another terminal A200 and 
two-way communication becomes possible, a send instruction to send the 
contents to the other terminal is sent together with the address information 
of the other terminal to the server A800 (Fig. 7: step S203). 
5 The server A800 sends the contents to the other terminal based on the 

address information of the other terminal that was received from the network 
terminal A100 (Fig. 7: step S204). 

The network terminals receive the contents from the server A800, and 
the applications of each of the network terminals read the contents (Fig. 

10 7 : step S205) . 

After the contents, or in other words, after the application data to 
be used in the presentation has been read by the network terminals at each 
location, the mode of the applications operated by the network terminals 
at each location is set (Fig. 3: step S102). 

15 The following processing is executed the same as in embodiment 1 (Fig. 

3: steps S103 to S109). 

With the cooperative application system and cooperative application 
method of the second embodiment described above, before the terminal on 
the receiving unit uses the contents, the contents are sent from the 

20 terminal on the sending side to a server, and the terminal on the receiving 
side receives the contents from the terminal on the sending side via the 
server. Therefore, the terminal on the sending side can send the contents 
to the server at a suitable time before the contents are needed at the 
conference or the like. 

25 In the second embodiment described above, an example was explained 

in which after the network terminal A100 calls another terminal A200 and 
two-way communication becomes possible, the network terminal A100 sends 
a send instruction together with the address of the other terminal to the 
server A800 in order to send contents to the other terminal, however, the 

30 timing for each of the network terminals to receive the contents from the 
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server A800 is not limited to this, and it can be set as desired as long 
as the operation of the applications at each location can be linked together 
before the presenter on the side of network terminal A100 uses the contents 
in a presentation, 

5 For example, the contents and the address of the network terminal at 

the distribution destination are sent beforehand from the network terminal 
A100 to the server A800, and the server A800 stores the contents in the 
contents-storage unit 801, and registers the address of the network 
terminal at the distribution destination in the user -management unit. 

10 Also, the network terminal A100 sends contents -acquisition information 
(here, this is address information for the server A800) to the other 
terminal. After the other network terminal A200 connects to the network 
700, it sends a send-contents request to the server A800. The server A800 
receives the send-contents request from the other network terminal A200 

15 and sends the user address from the user-information-management unit and 
the corresponding contents. 

By doing this, after each network terminal connects to the network 
700 and is in a state capable of communication, each network terminal can 
receive the contents by sending a send-contents request to the server A800, 

20 so anytime after the contents to be used are stored in the server A800, 
it is possible to receive the contents at a time suitable to the conditions 
at each location. 

Also, in the second embodiment described above, the case in which the 
network terminals A100, A200, A300 have the same construction and function 

25 as the network terminals 100, 200, 300 of the first embodiment was explained, 
however, the terminals do not have to be a special terminal for conferencing 
such as a network terminal, and any terminal that is constructed such that 
is can achieve the functions described above,- such as a personal computer, 
could be used. 

30 (Embodiment 3) 
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In the third embodiment of the invention, a cooperative application 
system that synchronizes the video, audio and application control is 
explained. 

Fig. 8 is a block diagram of the cooperative application system of 
5 this third embodiment of the invention. Also, Fig. 9 is a block diagram 
of the time-control unit of this third embodiment of the invention. 

Network terminal B100 is located on the side of the presenter, and 
network terminals B200 and B300 are located at each of the conference 
locations. Also, each of the network terminals B100, B200, B300 have the 
10 same functions as the network terminals 100, 200, 300 in the first 
embodiment, with only the time-control units 113, 213, 313 being added 
to the terminals. Here, any redundant explanation of the functions that 
are the same as those already explained for the first embodiment will be 
omitted. 

15 Here, Fig, 8, Fig. 9 and Fig. 10 will be used to explain the processing 

by the time-control units. 

The time-control units 113, 213, 313 that are shown in Fig. 8 and Fig. 
9 comprise: a multiplexed-data- generation unit 113A (213A, 313A) , and a 
reproduction- timing-control unit 113B (213B, 313B) . The 

20 multiplexed-data-generation unit 113A (213A, 313A) generates multiplexed 
packet data that synchronizes the video signal, audio signal and 
application-control signal with a specified synchronization signal. The 
application-control signal is a signal that transmits instructions to the 
application. Also, the reproduction- timing-control unit 113B (213B, 

25 313B) divides up the packet data received from a terminal on the sending 
side into a video signal, audio signal and application-control signal, 
and performs control such that the video, audio and application control 
are synchronized. 

Here, the synchronization signal is handled as one channel, however, 

30 as in the bit -multiplexing method, when generating multiplexed data, it 
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is possible to embed the synchronization signal for indicating the frame 
divisions in a fixed period, and then detect this synchronization signal 
on the receiving side and identify which media of the video signal, audio 
signal and application-control signal the synchronization signal and bits 
5 correspond to, and perform synchronization. 

The video-input units 114, 214, 314 shown in Fig, 8 input video to 
the terminal from a digital camera or network camera (not shown in the 
figure) that is located on the side of the terminal. 

When a terminal operates as a terminal on the sending side, the 

10 multiplexed-data-generation unit 113A, 213A, 313A shown in Fig. 9 
generates packet data from the multiplexed video signal from the 
video -input unit 114, audio signal form the microphone 108 and 
application-control signal that is output from the application-control 
unit 102, and sends it to the sending unit 103, 203, 303. 

15 When a terminal operates as a terminal on the receiving side, the 

reproduction-timing-control unit 113B, 213B, 313B separates the packet 
data received by the receiving unit 104, 204, 304 into a video signal, 
audio signal, application-control signal and synchronization signal, and 
reproduces and outputs the synchronized video, audio and application data. 

20 Figs. 10 shows the timing of the video signal, audio signal, 

application-control signal and synchronization signal of the cooperative 
application system of this third embodiment of the invention. Fig. 10A 
is a timing chart for the terminal on the side of the presenter, and Fig. 
10B is a timing chart for the terminals other than that of the presenter. 

25 The video signal A shown in Fig. 10A is input from a moving image 

or still image input from the digital camera or network camera (not shown 
in the figure) that is connected to the video-input unit 114 of the network 
terminal B100, which is the terminal on the side of the presenter. The 
audio signal shown in Fig. 10A is input from the microphone 108 of the 

30 network terminal B100. The application-control signal shown in Fig. 10A 
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is a signal for controlling the application in the network terminal B100, 
and it is output from the application-control unit 102. The 
synchronization signal A shown in Fig. 10A is a signal that becomes the 
reference for synchronizing the video signal A, audio signal A and 
5 application- control signal A in the network terminal B1000, and it is 
generated inside the network terminal B100. 

The video signal B shown in Fig. 10B is input from moving images or 
still images from a digital camera or network camera (not shown in the 
figure) that is connected to the video input unit 214, 314 of network 

10 terminal B200, B300 that is a terminal other than that of the presenter. 
The audio signal B shown in Fig. 10B is input from the microphone 208, 
308 of the network terminal B200, B300. The synchronization signal B shown 
in Fig. 10B is the signal that will be the reference when synchronizing 
the video signal B and audio signal B in the network terminal B200, B300, 

15 and it is generated in the network terminal B200, B300. 

First, in the network terminal B100 that operates as the terminal on 
the side of the presenter, the multiplexed-data-generation unit of the 
time-control unit 113 synchronizes each of the signals shown in Fig. 10A 
and generates packet data of the multiplexed data, then sends it outside 

20 the terminal by way of the sending unit 103. 

The network terminal B200, B300, which is a terminal other than that 
of the presenter, receives the packet data, and the 
reproduction-timing-control unit 213B, 313B of the timing-control unit 
213, 313 synchronizes and outputs (reproduces) the data. In other words, 

25 the network terminal B200, B300 displays the received video signal A on 
part of the display unit (not shown in the figure), reproduces the audio 
signal A by the .speakers 107, and sends an instruction to the application 
based on the application control signal. As a result, the application that 
receives the instruction displays the application data on part of the 

30 display unit (not shown in the figure). 
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On the other hand, in the network terminal B200, B300 that is a terminal 
other than that of the presenter, the multiplexed-data-generation unit 
of the time-control unit 213 , 313, synchronizes each of the signals shown 
in Fig. 10B and generates multiplexed packet data, then sends it outside 
5 the terminal by way of the sending unit 203, 303. At this time, network 
terminals B100, B300 (or B200) that are not the network terminal B200 (or 
B300) on the sending side, receive the packet data, and the 
reproduction- timing-control unit 113B, 313B (or 213B) of the time-control 
unit 113, 313 (or 213) synchronizes the data and outputs it. In other words , 

10 the network terminals Bl 00 , B300 (orB200) display the received video signal 
B on part of the display unit (not shown in the figure), and reproduce 
the audio signal B by the speakers 107, 307 (or 207). 

In this way, any of the terminals other than that of the presenter 
can send its own video signal B and audio signal B to another terminal. 

15 It is preferred that the terminal on the receiving side be constructed 
such that it is capable of switching among the video and audio received 
from each of the terminals. That is, in the case in which a presentation 
is being given by the presenter, for example, the video that is displayed 
on the display unit (not shown in the figure) and the audio that is 

20 reproduced by the speakers 107, 307 (or 207) , and when there is an opinion 
or question from the side of another terminal that reproduces the signals 
of the terminal on the side of the presenter, the 
reproduction-timing-control unit switches to the video and audio of the 
other terminal . 

25 As explained above, with this third embodiment of the invention, 

multiplexed data comprising an application-control signal and 
synchronization signal in addition to the video signal and audio signal 
are sent and received as packet data between terminals. Therefore, the 
amount of screen data of an application that is displayed by the terminal 

30 on the side of the presenter and that is sent to another terminal can be 
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reduced when compared with the conventional method of sending data, and 
it is also possible to reduce the processing for receiving packet data 
by other terminals, so even at terminals other than that of the presenter, 
it is possible to control the application (switch the presentation 
5 documents) while reproducing video and audio at the same timing as the 
terminal on the side of the presenter. Moreover, in the case where the 
video is high-definition quality, it is particularly possible to reduce 
the processing of sending and receiving packet data by the terminals, so 
a remarkable effect can be seen in the shift of the video and audio. 

10 Also, the reproduction-timing-control unit can switch the video and/or 

audio according to the input packet, so it is possible to know the condition 
a other receiving locations in addition to the condition at the side of 
the presenter. 
(Industrial Applicability) 

15 The network terminal, cooperative application system, cooperative 

application method and program of this invention make it possible to share 
video at various locations without sending the video of the display or 
the like of the sending side. Therefore, it is useful as a multimedia 
conference system terminal when a plurality of users at different locations 

20 participate in the same conference by way of a normal subscriber telephone 
line, Internet network, DSL network, private -line network or the like. 
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